PackageDescription: Transducers Data Flow API
Transducers Data Flow APILast published: April 11, 2023 by 'stm'
Defines 0 Classes
Extends 11 Classes
Transducers Data Flow API provides methods to process collections, streams and reducibles in a data flow style.
# Overview
Processing a sequence of data can often be seen as a data flow.
The operator #<~ creates a flow from a data source through a number of processing steps to a drain.
For example:
squares := Set <~ 20 take <~ #squared map <~ (1 to: 1000).
fileOut writeStream <~ #isSeparator filter <~ fileIn readStream.
Both examples use #<~ only to set up the data flow using reducing functions and transducers.
We have a clear separation of reading data (sources), processing elements (transducers) and writing data (drains).
#<~ is assoicative and each supexpression from the examples encapsulates a part of the flow, e.g.
- (Set <~ 20 take) is a drain that collects 20 elements from a source in a new Set.
- (20 take <~ #squared map) is a transducer that takes 20 squares from a source of numbers.
- (#squared map <~ (1 to: 1000)) is a source that provides the squares of the numbers 1 to: 1000.
## Sources aka Reducibles
Sources know how to provide a sequence of elements one-by one to a reducing function, e.g., via #reduce:init:.
All reducibles can act as data source, e.g.,
- Collection
- Stream
- BlockClosure that provides a new element on each evaluation
## Transducers
Transducers know how to process single elements. See the package Transducers for more information.
## Drains
Drains know how to collect data using a reducing function.
The following types can act as data source:
- Collection
- Stream
- Reduction which binds an initial value to a reducing function (a block or symbol)
# Usage by Example
1. We follow the example from the transducers package which
builds a coin flipping experiment and counts the occurrence of heads and tails.
| scale label sample result coin protocol |
scale := [:x | (x * 2 + 1) floor] map.
label := #(heads tails) replace.
sample := 1000 take.
"We use #<~ to create a data flow that executes the experiment."
result := Bag accumulate <~ sample <~ label <~ scale <~ Random new.
"We can also split the flow in (arbitrary) logical parts that can be re-used in other experiments."
coin := label <~ scale <~ Random new.
protocol := Bag accumulate.
result := protocol <~ sample <~ coin.
2. We build two sets containing negated respectively squared odd number from 1 to: 10.
| odds negatedOdds squaredOdds |
odds := #odd filter <~ (1 to: 10).
negatedOdds := Set <~ #negated map <~ odds.
squaredOdds := Set <~ #squared map <~ odds.